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High-level  commanders  and  executives  are  expert  problem  solvers,  but  we 
understand  very  little  of  the  nature  of  their  expertise.  We  know  that  the  executives 
have  considerable  knowledge  and  skill  that  enables  them  to  perform  their  tasks  in  an 
expert  manner,  but  we  understand  very  little  about  the  nature  of  this  knowledge  and 
these  skills.  The  skills  in  particular  are  poorly  conceptualized.  We  know  they  are  not 
the  kind  of  behavioral  skills  that  psychologists  have  traditionally  studied,  such  as  riding 
a  bicycle  or  shooting  a  rifle.  They  are  “cognitive”  skills;  they  involve  manipulation  and 
use  of  the  elements  of  domain  knowledge  for  some  purpose,  to  some  end. 

A  skill  is  defined  as  an  ability  to  do  something  well,  and  a  cognitive  skill  is  thus 
defined  as  an  ability  to  perform  a  cognitive  task  well.  A  cognitive  task  is  one  in  which 
successful  performance  depends  primarily  on  the  possession  and  skillful  manipulation 
of  information  and  knowledge;  the  product  of  a  cognitive  task  is  usually  cognitive  as 
well  —  an  idea,  a  plan,  a  decision,  a  solution  to  a  problem.  In  the  domain  of  executive 
performance  by  Army  commanders,  problem-solving  may  be  considered  the  generic 
cognitive  task,  encompassing  the  formal  mission-planning  tasks  and  the  less-clearly 
defined  decision-making  tasks  of  Army  command  and  control. 

In  addition  to  cognitive  skills,  executives  seem  to  possess  even  higher-level 
skills  that  enable  them  to  use  their  cognitive  skills  effectively.  Called  “metacognitive” 
skills,  these  are  defined  as  abilities  to  monitor  and  direct  the  operation  of  cognitive 
skills  to  obtain  the  greatest  possible  success.  Consider  the  example  of  what  is 
perhaps  the  greatest  cognitive  skill  of  humans,  the  ability  to  construct  sentences  to 
convey  meaning  (that  is,  language).  Metacognitive  skills  in  the  language  domain 
include  the  use  of  one's  knowledge  of  grammar  or  the  lexicon  to  form  more  effective 
sentences,  monitoring  the  response  of  the  listener  to  diagnose  communication 
success,  and  knowing  when  a  picture  is  worth  a  thousand  words. 

THEORIES  OF  METACOGNITIVE  SKILLS 

Metacomponents  of  intelligence.  Psychological  theories  have 
conceptualized  metacognition  in  a  variety  of  ways.  The  most  prominent  approaches 
have  focused  .on  the  executive-process  aspect  of  metacognitive  skills,  that  is,  the 
ability  to  organize,  sequence,  and  monitor  cognitive  processes  for  maximum 
effectiveness.  Sternberg’s  (1984, 1985)  triarchic  theory  of  intelligence  is  the  best 
example.  In  the  componential  subtheory,  three  types  of  components  (intellectual 
processes)  are  defined,  one  of  which  is  metacomponents.  "Metacomponents  are 
higher-order  executive  processes  used  in  planning,  monitoring,  and  evaluating  one’s 
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problem  solving"  (Sternberg,  1988,  p.  132).  The  planning  (or  "legislative") 
metacomponents  are  particularly  relevant  to  this  proposal.  Three  important 
metacog nitive  tasks  are  influenced  by  planning  metacomponents: 

1.  Recognizing  the  existence  of  a  problem. 

2.  Defining  the  problem. 

3.  Formulating  a  strategy  and  a  mental  representation  for  problem 
solution. 

Recognizing  the  existence  of  a  problem  sounds  easy  enough,  but  in  fact  it  is  a 
skill  that  vanes  considerably  among  people,  and  it  is  highly  correlated  with  intelligence 
and  creativity  (Sternberg,  1988).  Intelligent  problem-solvers  not  only  recognize  that  a 
problem  exists,  they  are  also  better  at  identifying  the  critical  problems  in  a  domain;  in 
the  words  of  one  researcher,  they  have  "good  taste"  in  problems  (Zuckerman,  1983). 
And  once  they  have  recognized  the  problem,  the  experts  define  the  problem  In  a  way 
that  makes  the  problem  soluble.  Then  they  formulate  a  strategy  that  promises  to  solve 
the  problem,  and  they  represent  the  problem  mentally  (or  in  a  computer  program)  in  a 
way  that  is  close  to  optimal  for  problem  solution. 

Piaget’s  stages  of  intellectual  development.  Jean  Piaget  has  a  theory  of 
intellectual  development  that  can  be  extended  to  the  adult  years.  Infants  are  said  to  be 
at  a  relatively  primitive,  sensory-motor  stage  of  intellectual  development  that,  with  the 
onset  of  speech,  becomes  a  conceptual-symbolic  stage  called  preoperational  (Fiavell, 
1 963).  Around  the  age  of  six,  children  enter  the  stage  of  concrete  operations,  in  which 
they  can  apply  operations  (mental  routines)  to  transform  information  in  some  way  — 
adding  two  numbers  to  get  a  third,  placing  all  red  objects  in  the  same  pile.  Around  the 
age  of  12,  children  begin  the  final  stage  of  intellectual  development  called  formal 
operations.  In  which  they  can  apply  mental  routines  to  abstract  material.  For  example, 
an  adolescent  can  solve  a  problem  like  "If  a  suitcase  can  eat  four  rocks  in  one  day, 
how  many  can  it  eat  in  two  days?"  Younger  children  cannot  imagine  a  suitcase  that 
eats  rocks,  so  they  will  refuse  to  answer  the  question;  they  cannot  disregard  the 
content  of  the  problem  (its  concrete  aspects)  and  reason  in  a  purely  hypothetical  way 
(using  the  form,  or  formal  aspects,  of  the  problem). 

The  advent  of  formal  reasoning  creates  an  interest  in  form,  that  is,  adolescents 
become  fascinated  by  the  formal  structures  and  processes  of  thought.  They  think 
about  thinking,  which  is  a  good  definition  of  metacognition.  One  of  the  products  of 
Piaget’s  theory  is  a  body  of  research  on  metacognition,  much  of  it  on  memory  or 
“metamemory”;  this  research  will  be  discussed  below. 

Adult  stages  of  metacognitive  development.  Piaget's  stages  have  been 
extended  to  the  adult  years  by  Schaie  and  Geiwitz  (1982).  The  development  of  formal 
operations  suggests  that  metacognitive  structures  and  strategies  can  be  applied  to 
direct  cognition  toward  problem  solution,  a  skill  that  continues  to  develop  throughout 
adulthood.  The  adult  stages  of  intellectual  development  reflect  a  general  increase  in 
metacognitive  skills,  which  underlie  adult  application  and  use  of  knowledge,  rather 
than  increases  in  cognitive  skills,  which  underlie  childhood  acquisition  of  basic 
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knowledge.  The  first  adult  stage,  which  occurs  in  young  adulthood,  is  temporal 
monitoring,  which  represents  the  application  of  intelligence  in  situations  that  have 
profound  consequences  for  achieving  long-term-goals  (involving  decisions  about 
career  and  marriage).  Temporal  monitoring  is  a  kind  of  quality  control  process  applied 
to  problem-solving  when  the  solutions  must  be  integrated  into  a  life  plan  that  extends 
far  into  the  future.  It  is  similar  to  skills  used  by  Army  commanders  when  they  prepare  a 
synchronization  matrix  for  the  various  Battlefield  Operating  Systems  in  a  mission  plan. 

A  second  major  application  of  Intellect  in  adulthood  occurs  in  the  second  adult 
stage,  called  social  rnonitoring.  Typically  this  stage  develops  when  a  family  is 
established,  and  the  individual  must  begin  monitoring  not  only  his  or  her  own 
behavior,  but  also  that  of  spouse  and  offspring.  Similar  extensions  of  monitoring  skills 
are  required,  as  responsibilities  for  others  are  acquired  on  the  job  and  in  the 
community.  Social  monitoring  includes  temporal  monitoring  of  a  group  of  people  who 
are  all  working  toward  the  same  end;  not  only  must  their  activities  be  synchronized  for 
m^imum  effectiveness,  but  metacognitive  skills  such  as  resource  allocation  and  the 
efficient  division  of  labor  among  group  members  become  primary  determinants  of 
group  performance.  In  Gardner's  theory  of  multiple  talents,  social-monitoring  skills  fall 
into  the  category  of  personal  intelligence  (Gardner,  1983),  which  Includes  the  ability  to 
take  another  person’s  perspective  in  a  training  situation;  Anne  Sullivan,  the  teacher  of 
Helen  Keller,  is  assumed  to  have  been  high  in  personal-social  intelligence  and  skills. 
In  the  military  domain,  social  monitoring  is  a  set  of  metacognitive  skills  that  will  serve  a 
commander  of  a  combined-arms  unit  well. 

A  third  adult  stage  of  intellectual  development  we  call  executive  monitoring. 
Many  individuals'  responsibilities  become  exceedingly  complex.  They  become 
presidents  of  business  firms,  deans  of  academic  institutions,  officials  of  churches,  or 
commanders  of  divisions  or  corps.  As  such,  they  need  to  understand  how  an 
organization  works:  the  structure  and  the  dynamic  forces,  who  answers  to  whom,  anch 
for  what  purpose.  They  must  monitor  organizational  activities  not  only  on  a  temporal 
dimensiori  (past,  present,  and  future)  but  also  up  and  down  the  hierarchy  that  defines 
the  organization.  Executive  monitors  must  know  the  plans  and  intentions  of  superiors, 
and  they  must  devise  a  structure  for  monitoring  and  controlling  the  implementation  of 
policies  at  the  lower  levels  of  responsibility. 

Metamemory.  As  mentioned  above,  Piaget’s  theory  led  to  research  on 
metacognitive  skills  in  memory,  or  metamemory.  The  specific  topic  in  which  we  are 
most  interested  is  problem  solving  or  what  we  might  call  meta-reasoning. 

Nevertheless,  the  research  on  metamemory  is  of  interest,  not  only  because  it 
represents  a  productive  approach  to  metacognitive  skills,  but  also  because  it  has  been 
a  developmental  approach,  which  offers  clues  to  the  development  of  metacognitive 
skills  in  general. 

John  Flavell  distinguishes  between  two  broad  areas  of  metamemory  skills 
(Flavell  &  Wellman,  1 977).  The  first  is  sensitivity  to  the  need  for  planful  memory.  At 
first,  the  need  for  planful  memory  may  be  explicitly  stated  by  a  teacher  or  parent,  who 
may  instruct  the  child  to  remember  something.  Later  the  child  may  apply  the 
metamemory  skills  spontaneously,  knowing  by  now  that  one  can  prepare  for  later 
retrieval,  that  there  is  a  difference  between  information  processing  for  later  recall  and 
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other  cognitive  processing  of  information.  The  second  broad  area  of  metamemory  is 
knowledge  of  variables  that  affect  memory  performance.  These  variables  include 
person  variables  (some  people  have  better  memories,  people  are  likely  to  forget 
information  learned  under  emotional  stress),  task  variables  (meaningless  information 
Is  harder  to  remember),  and  strategy  variables  (rehearsal  is  a  good  mnemonic 
strategy).  For  example,  children  learn  that  if  one  variable,  say  task  difficulty,  is  high, 
predicting  poor  memory,  they  must  compensate  with  another  variable,  allocating  more 
study  time.  The  child  may  test  memory  and  then  concentrate  rehearsal  on  the 
unlearned  items. 


Figure  1.  A  theory  of  metamemory.  (From  Nelson  &  Narens,  1990) 
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Nelson's  theory  of  metamemory,  although  limited  to  one  kind  of  cognition,  is 
another  framework  for  our  analysis  (Nelson  &  Narens,  1 990).  Rgure  1a  depicts  the 
stages  of  memory  —  acquisition,  retention,  and  retrieval  —  and  gives  examples  of  both 
the  monitoring  and  control  functions  of  metamemory  skills.  The  monitoring  functions 
determine  the  control  functions  to  be  activated,  as  shown  in  Figures  1b  and  1c.  In 
Figure  1b,  people  are  asked  to  learn  some  material  to  a  certain  criterion  —  "Learn  this 
list  of  CIA  agents  perfectly,  then  destroy  the  list"  —  and  monitors  list  acquisition  until 
they  make  a  metacognitive  "judgment  of  learning"  (JOL)  or  experience  a  "feeling  of 
knowing"  (FOK).  If  the  JOL  and  FOK  Indicate  that  more  study  is  required,  control 
functions  are  activated;  the  kind  of  learning  strategy  is  selected  (rote  memorization, 
use  of  mnemonic  devices),  and  study  time  Is  allocated  to  the  individual  Items  on  the 
list,  according  to  the  metacognitiveiy  perceived  need.  Figure  1  c  show  the  same 
relations  in  a  retrieval  task,  where  FOK  and  one's  confidence  in  retrieved  answers 
(CRA)  determines  search  strategy  and,  ultimately,  termination  of  the  search. 

METACOGNITIVE  SKILLS 

Metacognitive  skills  are  skills  related  to  the  effective  use  of  cognitive  skills,  by 
using  them  strategically  and  monitoring  and  controlling  their  execution  In  the 
environment.  We  will  focus  on  the  cognitive  skill  of  problem  solving,  a  skill  of 
significant  importance  to  the  US  Army  and  one  that  is  often  studied  in  the  context  of 
metacognitive  control. 

The  following  discussion  lists  some  of  the  metacognitive  skills  that  have  been 
identified  in  the  context  of  problem  solving.  All  of  these  skills  —  their  identity,  their 
function  —  are  controversial,  that  is,  there  is  continuing  debate  on  all  of  them,  and 
none  is  generally  accepted.  We  believe  each  identifies  at  least  an  area  of 
metacognitive  impact,  if  not  the  precise  means  of  influence.  The  purpose  of  our 
conceptual  model,  which  itself  is  a  “work  in  progress,"  is  to  clarify  these  skills  and  their 
interrelationships. 

Detection  of  a  problem.  Does  a  problem  exist?  One  metacognitive  skill  is 
the  ability  to  detect  a  problem  when  it  arises.  Somehow  the  individual  must  monitor 
the  discrepancies  between  the  current  state  and  the  goal  state,  noting  a  problem  when 
the  discrepancies  exceed  a  certain  value. 

Representation  of  a  problem.  Expert  problem  solvers  seem  to  have  this 
metacognitive  skill  in  high  degrees,  and  well  they  should,  because  evidence  indicates 
that  it  is  a  critical  skill  for  effective  solutions.  How  an  individual  states  the  problem  is  a 
prime  determinant  of  success  in  solving  it. 

Selection  of  a  problem-solving  method.  There  are  many  ways  to  solve 
problems.  Good  problem  solvers  know  many  methods,  and  they  have  the  ability  to 
select  wisely,  given  certain  characteristics  of  the  problem  domain. 

Strategic  application  of  problem-solving  methods.  Good  problem 
solvers  have  strategies  for  solving  the  problem.  They  apply  a  potentially  effective 
method,  constantly  monitoring  the  changes  in  problem  state  that  the  method  produces 
to  see  if  a  solution  has  occurred.  They  know  what  they  will  try  next,  and  why. 
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Evaluation  of  solution  candidates.  Like  the  wargame  evaluation  of  the 
three  Courses  of  Action  selected  in  the  mission-planning  process,  good  problem 
solvers  evaluate  potential  solutions,  to  see  if  the  discrepancy  between  goal  state  and 
current  state  has  been  reduced. 

Recognition  of  errors.  Good  problem  solvers  spot  errors  more  quickly  and 
more  accurately  than  poor  problem  solvers.  Common  errors  that  result  from  cognitive 
biases  and  misapplied  heuristics  of  the  sort  studied  by  Tversky  and  Kahneman  (1974) 
are  anticipated  and  guarded  against. 

Resource  allocation.  Good  problem  solvers,  when  they  identify  a  problem, 
can  allocate  their  problem-solving  resources  to  create  the  most  advantageous 
environment  for  the  solution  to  the  problem.  If  the  problem  requires  memory,  for 
example,  they  know  how  long  It  will  take  to  memorize  the  material,  and  they  allocate 
the  time  accordingly. 

Temporal  monitoring.  Temporal  monitoring  includes  the  effective  and 
strategic  allocation  of  time  resources,  but  it  also  includes  the  monitoring  function,  to 
see  if  the  solution  is  developing  "according  to  schedule."  Successful  managers  are 
noted  for  their  ability  to  maximize  the  effective  use  of  their  time  (Bray  &  Howard,  1981). 
In  complex  problems,  many  resources  must  be  synchronized  for  maximum  impact. 

Social  monitoring.  Problem  solving  in  a  social  context  —  that  is,  most 
problem  solving  —  is  different  from  the  same  activity  in  isolation.  Good  problem 
solvers  allocate  human  resources  wisely,  and  they  try  to  establish  a  social 
environment  in  which  the  group  can  function  effectively.  This  means,  among  other 
things,  they  have  to  take  personalities  into  account,  watch  for  conflicts,  and  moderate 
disputes.  To  manage  effectively  they  must  have  sensitivity  and  understanding  of  other 
peoples'  perspectives  and  goals. 

The  social-emotional  aspects  of  leadership  became  more  important  in 
American  businesses  as  big  corporations  changed  from  family-owned  enterprises 
controlied  by  autocratic  individuals  to  publicly  owned  corporations  led  by  committees. 
Psychologists  were  brought  in  to  advise  corporations  on  how  best  to  solve  problems  in 
groups  (Geiwitz,  1980).  It  was  soon  discovered  that  lack  of  knowledge  and  logic  was 
not  the  chief  impediment  to  effective  group  solutions;  interpersonal  relationships  were 
much  more  crucial.  One  member  of  a  committee  would  suggest  a  perfectly  logical 
solution  to  a  problem,  but  the  group  would  reject  it,  because  they  disliked  him  or  her. 
Thus,  the  original  groups  of  executives  brought  together  to  learn  how  to  solve 
problems  —  called  "training  groups"  or  "T-groups"  —  were  soon  supplanted  by 
"sensitivity  T-groups"  and,  later,  "encounter  groups."  Members  of  these  groups 
learned  how  to  recognize  the  emotional  reactions  their  actions  provoked  in  other 
people,  usually  through  interaction  and  interpretation. 

Executive  monitoring.  Executives  have  key  positions  in  a  hierarchical 
network  of  individuals.  To  be  effective  problem  solvers,  they  must  understand  their 
position  in  the  network:  their  relationship  with  higher  authorities,  their  relationship  with 
subordinates,  their  relationship  with  peers.  Executive  monitoring  is  more  than 
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temporal  and  social  monitoring,  although  it  includes  these  lesser  skills.  In  one  study, 
for  example,  the  major  difference  between  young  executives  and  older,  more 
experienced  executives  was  the  greater  ability  of  the  older  executives  to  market  for  the 
company  (Schaie  &  Geiwitz,  1982).  Marketing  is  a  very  high  level  skill,  involving 
knowledge  of  what  the  company  is  trying  to  do,  good  perception  of  the  needs  of  a 
potential  customer,  and  a  good  sense  of  the  company  capabilities  to  solves  certain 
kinds  of  problems.  It  usually  develops  slowly  over  the  lifespan,  and  many  executives 
never  become  accomplished  at  business  development. 

A  CONCEPTUAL  MODEL  OF  METACOGNITIVE  SKILLS 

Problem  Solving  (A  Cognitive  Skill) 

To  develop  a  model  of  how  metacognitive  skills  augment  and  facilitate  the 
cognitive  skill  of  problem  solving,  we  need  first  a  model  of  the  problem-solving  task. 
Complete,  theory-bound  models  of  problem  solving  do  not  exist,  but  the  general 
stages  and  principles  have  been  described  by  rnany;  the  Command  Estimate  is  based 
on  such  descriptions.  Problems  are  “initial  states,”  their  solutions  are  “goal  states,” 
and  problem  solving  methods  are  means  for  transforming  the  initial  state  into  a  goal 
state  (Newell  &  Simon,  1972).  Problem  solving  in  the  business  world  (therein  called 
management)  has  been  described  by  Kepner  and  Tregoe  (1965)  and  Plunkett  and 
Hale  (1982)  in  similar  terms;  the  following  steps  are  a  combination  of  the  two  major 
managerial  descriptions; 

Step  1 :  Identify  the  problem.  Compare  actual  performance  with  expected 
performance,  and  define  the  problem  as  the  discrepancy  between  them. 

Step  2:  Describe  the  problem.  What  objects  are  involved?  What  is 
wrong?  Where  is  the  problem  occurring?  When  did  the  problem  begin?  What 
is  the  extent  of  the  problem? 

Step  3:  Identify  the  cause  of  the  problem.  Compare  similar  situations 
with  and  without  problems.  Compare  affected  and  unaffected  objects. 

Determine  other  differences  between  situations  and  objects,  to  identify  potential 
causes. 

Step  4:  Solve  the  problem.  Eliminate,  modify,  or  insulate  the  cause  of  the 
problem  so  that  actual  and  expected  performance  again  coincide. 

The  Command  Estimate  is  another  general  description  of  the  problem  solving 
process.  After  analysis  of  the  mission  and  the  orders,  relevant  information  about  the 
situation  (including  the  terrain  of  the  battlefield)  and  the  enemy  is  collected,  in  effect 
identifying  and  describing  the  problem  and  the  goal  state  (the  mission  objectives).  A 
number  of  actions  that  might  solve  the  problem  are  described  —  Courses  of  Actions 
(COAs).  Each  COA  is  played  out,  step  by  step,  in  a  wargame  technique  and  evaluated 
in  terms  of  several  mission  objectives.  One  COA  is  recommended,  but  all  are  briefed 
to  the  commander,  who  makes  the  final  decision.  The  Command  Estimate,  therefore, 
describes  in  more  detail  Step  4  above,  suggesting  that  the  problem  solver  generate 
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options,  evaluate  these  options,  and  then  choose  the  option  with  the  greatest  apparent 
likelihood  of  success.  These  skills,  however,  may  be  better  considered  metacog nitive. 

Technical  Monitoring  and  Controi 

As  a  first  approximation  of  a  conceptual  model  of  metacognitive  skills,  which  we 
expect  to  evolve  as  library  and  empirical  research  continues,  we  will  focus  on  those 
skiiis  that  facilitate  the  technical  activities  in  problem  solving.  The  technical  aspects  of 
problem  solving  comprise  the  purely  formal  operations  designed  to  identify,  represent, 
and  solve  the  problem,  in  addition  to  the  technical  aspects,  there  are  temporal,  social, 
and  organizational  aspects  of  the  problem-solving  process,  no  less  important  in  many 
cases;  we  will  discuss  these  aspects  in  a  later  section. 

Rgure  2  tries  to  align  the  metacognitive  skiiis  relevant  to  monitoring  and 
controlling  the  technical  process  of  problem  solving.  This  modei,  or  submodel,  looks 
suspiciously  like  a  practical  description  of  the  scientific  method  —  as  it  should, 
perhaps,  since  the  scientific  method  is  the  best  problem-solving  method  known  to 
humans.  Persons  with  a  high  degree  of  these  metacognitive  skiiis  (with  comparable 
domain  knowledge)  would  be  called  experts  (Laskey,  Leddo,  &  Bresnick,  1990). 
Usually  skilled  executives  or  high-level  commanders  are  expert  in  this  sense  (or  were 
at  one  time,  in  lower-level  positions),  but  they  have  other  skills  as  well,  and  these  skills 
are  also  pertinent  to  executive  problem  solving. 

Temporal,  Social,  and  Organizational  Monitoring  and  Control 

Technical  skills,  cognitive  or  metacognitive,  are  better  conceptualized  than 
temporal,  social,  and  organizational  skills,  although  the  latter  are  of  critical  importance 
in  the  assessment  and  training  of  executive  performance.  What  we  present  in  this 
section  is  by  no  means  an  adequate  model  of  these  nontechnical  skills,  but  rather  a  - 
scaffolding  for  the  later  construction  of  such  a  model.  We  have  some  comments  on 
what  these  models  might  look  like,  at  the  end  of  this  section. 

Temporal  skills.  As  defined  previously,  temporal  skills  have  to  do  with  the 
effective  use  of  time  in  problem  solving.  Scheduling  is  one  such  skill,  one  that  enables 
the  executive  to  ailocate  temporal  resources  effectively  to  complete  the  task  in  the 
allotted  time.  For  example,  the  Army  has  developed  the  “1/3, 2/3  rule”  for  mission 
planning,  that  is,  a  commander  at  any  echelon  should  use  1/3  of  the  total  time  before 
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Figure  2.  A  model  of  metacognitive  skills  influencing  technical  problem  solving. 

the  beginning  of  mission  execution  for  his  planning  and  leave  2/3  for  subordinate 
commanders  to  do  their  planning.  Another  important  temporal  skill  is  synchronization 
(Long,  1989).  In  Army  practice,  tactical  commanders  are  taught  to  construct  a 
synchronization  matrix  (Tactical  Commanders  Development  Course  at  Fort 
Leavenworth),  in  which  the  temporal  aspects  of  the  actions  (start,  stop,  etc.)  of  each  of 
the  seven  Battlefield  Operating  Systems  (BOS)  are  charted,  to  ensure  maximum 
Impact  of  the  operation  as  a  whole.  In  most  complex  problem  solving,  several 
resources  must  be  synchronized. 

Social  skills.  Effective  leaders  allocate  human  resources  wisely  and 
delegate  responsibility  in  a  way  that  satisfies  the  technical,  temporal,  and 
organizational  requirements  of  the  task.  Studies  of  leaderless  groups  show  that  two 
types  of  leaders  typically  emerge:  a  task  leader,  who  facilitates  the  technical  aspects  of 
problem  solving  in  a  group,  and  a  socioemotional  leader,  who  facilitates  the  social 
aspects.  There  are  several  social  skills  relevant  to  problem  solving.  One  is  the  ability 
to  motivate  subordinates  and  to  use  rewards  (and  possibly  punishments)  effectively. 
Laskey  et  al.  (1 990)  speak  of  the  importance  of  shared  ownership,  that  is,  an  effective 
leader  takes  all  points  of  view  into  account  and  develops  a  consensus  in  which  ail 
participants  consider  themselves  to  have  contributed  to.  Conflict  management  is 
another  important  social  skill. 

Organizational  skills.  In  a  previous  section,  we  referred  to  organizational 
skills  as  executive  monitoring.  Executives  have  key  positions  in  a  hierarchical  network 
and,  to  be  effective,  must  understand  their  position  and  its  relationship  with 
superordinates,  subordinates,  and  peers.  Executives  must  know  the  long-term  goals 
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of  the  organization  and  how  such  goals  are  achieved  in  the  context  of  the 
organizational  structure.  Executives  are  especially  skilled  at  organizational 
development,  which  in  the  case  of  business  organizations  means  business 
development;  in  the  Army,  organizational  development  means  the  creation,  equipping, 
training,  and  fielding  of  military  units,  while  exploring  new  technologies,  training 
methodologies,  and  other  innovations  that  might  lead  to  more  effective  armies  in  the 
future. 


The  conceptual  model.  Temporal,  social,  and  organizational  skills  are 
clearly  important  to  effective  executive  performance,  but  they  have  rarely  been 
modeled.  Even  more  rarely  have  these  metacognitive  skills  been  modeled  in  the 
same  context  as  technical  problem-solving  skills.  One  possibility,  which  we  will 
explore,  is  a  generic  model  like  the  one  presented  In  Figure  2  for  technical  problem 
solving.  Perhaps  the  same  model  can  be  used  for  the  four  different  sets  of 
metacognitive  skills:  technical,  temporal,  social,  and  organizational.  For  temporal 
skills,  the  problems  would  most  likely  have  a  time  line  —  ‘^Ne  are  going  to  run  out  of 
ammunition  around  1600  hours.”  The  solutions  would  also  be  time-based;  in  many 
cases,  they  would  be  scheduling  solutions  or  synchronization  solutions.  Allocation  of 
resources  over  time  would  also  contribute  to  the  solutions. 

Similarly,  the  social  skills  would  aid  in  the  solution  of  social  problems  that  are 
preventing  the  executive  from  reaching  organizational  goals.  The  technical  problem 
may  have  been  solved,  but  the  executive’s  subordinates  are  reluctant  to  execute  the 
solution  because  of  fear  or  fatigue.  The  social  solutions  would  be  of  the  sort  that 
reduce  conflict  in  the  group  and  motivate  the  group  members  to  work  toward  the 
group’s  goals.  Organizational  skills  would  aid  in  the  solution  of  organizational 
problems,  e.g.,  if  the  organizational  structure  is  such  that  no  one  has  responsibility  for 
certain  subtasks,  an  organizational  solution  might  assign  such  responsibility  in  an  ad 
hoc  fashion,  to  facilitate  goal  attainment.  I  am  reminded  of  my  boss  on  a  road  crew  I  - 
once  worked  for:  Working  long  hours  in  the  hot  summer,  the  workers  complained  of 
boredom  and  lack  of  motivation;  they  proposed  that  jobs  be  rotated,  so  that  one  day  I 
might  clean  the  road  in  advance  of  the  seal-coating  unit,  the  next  day  I  might  drive  a 
packer,  and  the  third  day  I  might  drive  a  gravel  truck.  A  social  motivational  problem. 
The  boss  said,  however,  that  the  proposed  solution  could  not  be  implemented 
because  of  a  conflicting  organizational  problem  that  would  arise  if  it  were:  No  one 
worker  would  have  responsibility  for  the  maintenance  of  his  piece  of  equipment,  which 
we  had  to  admit  meant  that  the  equipment  would  surely  fall  into  disrepair. 

TRAINING  AND  ASSESSMENT 

The  purpose  of  the  conceptual  model  is  to  guide  training  and  assessment  of 
metacognitive  skills  in  executive-level  commanders.  To  do  so,  the  model  must  include 
training  and  assessment  agents  and  objects  derived  from  and  related  to  the  model  of 
the  metacogniJively  influenced  task;  the  model  must  have  training  and  assessment 
modules.  This  section  is  a  place  holder  for  those  modules,  which  will  be  developed 
during  the  course  of  the  project  research.  As  with  the  temporal,  social,-and 
organizational  skills,  we  will  suggest  possible  answers  to  the  questions  of  how  we  can 
train  metacognitive  skills  and  how  we  can  assess  them  In  experienced  commanders. 
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TRAINING 


Most  of  the  psychological  research  on  metacognition  has  heretofore  been 
developmental,  focused  on  childhood  changes  with  age  and  experience.  This 
research  gives  some  clues  as  to  the  ways  such  skills  can  be  trained,  but  unfortunately 
the  focus  has  been  on  when  (at  what  age)  such  skills  develop  and  what  effect  such 
skills  have  on  children’s  performance  (enhanced)  in  intellectual  tasks.  Case  (1984), 
for  example,  describes  ten  levels  of  intellectual  skill  development,  the  last  three  or  four 
of  which  are  skills  involving  abstractions  similar  to  the  metacognitive  processes 
described  in  the  conceptual  model.  These  skills  presumably  develop  in  the  teens,  with 
the  ability  to  do  “abstract  mapping”  observed  in  most  cases  between  14  and  16  years. 
Case  says  little  about  how  these  skills  develop  —  some  may  require  neurological 
development.  Case  does  hypothesize  that  the  limited  processing  capacity  of  humans 
is  first  devoted  to  basic  operations  and  then,  as  these  (cognitive)  skills  develop  and 
require  less  conscious  control,  more  of  the  capacity  can  be  devoted  to  metacognitive, 
support  skills.  Sternberg  (1984)  has  similar  ideas,  describing  a  process  in  which 
crude  metacognitive  skills  (metacomponents)  are  used  to  control  intellectual 
operations,  receive  and  interpret  feedback  on  the  results  of  such  operations,  and 
refine  themselves  on  the  basis  of  that  feedback.  In  Sternberg’s  theory,  “the 
metacomponents  form  the  major  basis  for  the  development  of  intelligence”  through 
continual  feedback  loops  (Sternberg,  1984,  p.  172).  If  the  metacomponents  are  not 
used  to  increase  metacognitive  skills,  significant  increases  in  intellectual  performance 
are  unlikely;  mere  experience  or  practice  will  not  be  effective. 

Metacognitive  skills  support  problem  solving  performance  in  abstract  ways, 
providing  a  general  framework  and  a  general  procedure.  Thus,  the  problem  solving 
methods  described  by  Kepner  and  Tregoe  (1965)  and  Plunkett  and  Hale  (1982)  can 
be  taught  as  metacognitive  skills.  In  such  training  programs,  students  are  taught  how 
to  detect,  define,  and  describe  the  problem,  and  then  how  to  identify  and  eliminate  the 
cause  of  the  problem.  These  training  programs,  in  a  sense,  teach  the  scientific  method 
for  problem  solving.  Thus,  science  training  can  be  examined  for  clues  as  to  how 
metacognitive  skills  can  be  taught.  Compared  to  Kepner-Tregoe  training,  science 
training  focuses  more  on  the  arrangement  of  events  to  determine  the  cause  of  the 
problem.  Another  focus  in  the  scientific  method  is  in  the  arrangement  of  events  so  that 
empirical  data  can  be  interpreted  meaningfully.  For  example,  random  assignment  of 
experimental  subjects  to  groups  for  later  comparisons  is  a  preferred  method,  because 
randomization  (with  large  numbers  of  subjects)  equalizes  all  variables  between 
groups  except  for  the  variable  of  interest  (the  independent  variable).  If  groups  are 
formed  on  the  basis  of  preexisting  characteristics  (comparing  men  with  women),  the 
groups  differ  in  an  infinite  number  of  ways  besides  the  variable  of  interest  (gender). 

Can  one  train  people  to  use  abstract,  domain-independent  inferential  rules  to 
think  about  important  events  in  their  lives?  A  surprising  number  of  theorists  say  no, 
that  Plato’s  doctrine  of  formal  discipline,  which  holds  that  the  study  of  abstract  rule 
systems  trains  the  mind  for  reasoning  about  concrete  problems,  is  invalid  (Thorndike, 
1906).  Thorndike  was  able  to  show  that  there  was  very  little  transfer  of  training  from 
one  course  of  study  (e.g.,  Latin)  to  other  courses.  If  his  view  is  correct,  we  will  have 
little  luck  training  metacognitive  skills.  But  current  work  on  this  issue,  exemplified  by 
the  research  of  Nisbett  and  his  colleagues  (Nisbett  et  al.,  1987),  suggests  that  abstract 
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skills  can  be  taught.  The  primary  problem  in  the  transfer  of  training  is  in  the  ability  of 
students  to  apply  the  abstract  rules  to  specific  domain  content.  This,  the  proper 
representation  of  the  problem  so  that  the  abstract  rule  can  be  seen  to  apply,  is  what 
needs  to  be  taught. 

Specific  metacognitive  skills  have  been  the  substance  of  specific  training 
courses,  and  experience  of  this  sort  also  illuminates  our  approach  to  training 
metacognitive  skills  in  general.  For  example,  synchronization  of  resources  has  been 
trained  at  Fort  Leavenworth  for  several  years,  in  a  precommand  course  called  Tactical 
Commander's  Development  Course  (TCDC).  Students  are  taught  a  general 
methodology  that  can  be  applied  to  specific  mission-planning  exercises  (Long,  1989). 

In  essence,  they  are  given  a  matrix  to  plan  the  activities  of  each  of  the  seven  Battle 
Operating  Systems  (BOSs)  along  a  time  line  that  begins  before  H-hour  and  continues 
into  sequel  missions.  They  are  given  extensive  practice  filling  in  the  matrix  in  a  variety 
of  mission-planning  exercises  and  encouraged  to  continue  the  practice  in  their 
command  roles.  Synchronization  is  a  metacognitive  skill  related  to  temporal 
monitoring. 

Social  monitoring  and  control  comprises  a  set  of  metacognitive  skills  related  to 
team  or  group  performance.  Salas  and  his  colleagues  have  been  developing  theory 
concerned  with  the  training  of  teamwork  skills,  which  has  obvious  implications  for 
training  metacognitive  skills  (e.g.,  Glickman  et  al.,  1987).  Rndings  include  the  fact  that, 
for  most  team  training,  training  in  a  team  context  is  superior  to  individual  training  on 
team  tasks.  Also,  three  distinct  factors  appear  in  team  training:  a  taskwork  factor, 
reflecting  training  on  the  team  task;  a  teamwork  factor,  reflecting  learning  to  coordinate 
and  communicate  within  the  team;  and  a  jelling  factor,  which  reflects  the  ability  to  put 
the  taskwork  and  the  teamwork  together  in  an  integrated  approach  to  problem  solving 
in  the  group  setting.  These  three  factors  are  distinct  at  the  beginning  of  training,  but 
converge  during  the  final  stages  of  training.  These  findings  bear  a  striking 
resemblance  to  the  concept  of  two  kinds  of  leadership;  task  leadership  and 
socioemotional  leadership. 

ASSESSMENT 

The  first  step  in  skill  assessment  is  a  thorough  task  analysis,  according  to  the 
standard  Instructional  Systems  Development  (ISD)  methodology  (Vineberg  &  Joyner, 
1980).  If  the  goal  is  to  assess  a  cognitive  skill,  one  must  do  a  cognitive  task  analysis. 
Our  goal  is  to  assess  a  metacognitive  skill;  what  then  is  required  of  us?  A 
metacognitive  task  analysis?  Procedural  or  behavioral  task  analysis  is  a  fairly  well- 
defined  technique  (Drury  et  al.,  1987),  and  even  cognitive  task  analysis  is  becoming 
more  common  and  more  standardized  (Lesgold  et  al.,  1990).  Methods  for  group 
cognitive  task  analysis  have  been  developed  (Salas,  1993).  But  I  know  of  no  work  on 
individual  or  group  metacognitive  task  analysis.  Before  we  can  begin  to  construct  tests 
of  metacognitive  skills,  we  must  first  develop  such  a  task-analysis  methodology.  The 
training  of  metacognitive  skills  also  depends  on  such  analyses. 

Once  the  appropriate  task  analysis  has  been  accomplished,  the  knowledge  and 
skills  (KSs)  required  to  perform  each  step  are  determined.  The  KSs  (not  the 
procedural  steps  of  the  task)  are  the  fundament  of  both  training  programs  and 
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assessment  devices.  Classroom  training  is  designed  to  provide  the  task  knowledge, 
whereas  laboratory  and  on-the-job  training  is  designed  to  teach  task  skills.  Verbal 
tests  are  designed  to  assess  task  knowledge  and  performance  tests  are  designed  to 
assess  task  skills.  So  the  question  before  us  is,  How  do  we  assess  metacognitive 
skills  and  knowledge  related  to  problem  solving? 

Cognitive  task  analysis  uses  knowledge-acquisition  techniques  (KATs)  to  elicit 
the  knowledge  and  the  covert  decision  processes  involved  in  cognitive  task 
performance  (Geiwitz,  Komell,  &  M’Closkey,  1992).  Many  KATs  also  seem 
appropriate  for  the  investigation  of  metacognitive  KSs.  Protocol  Analysis,  for  example, 
has  a  domain  expert  “think  out  loud”  while  performing  the  task  (Ericsson  &  Simon, 
1984).  If  the  expert  were  primed,  not  to  describe  the  direct  problem-solving  processes, 
but  to  describe  the  goals  of  the  endeavor  and  the  strategy  for  goal  attainment,  we 
might  elicit  the  metacognitive  steps  that  monitor  and  control  the  direct  processes. 

These  steps  could  then  be  analyzed  for  required  KSs  in  a  conventional  manner. 
Similarly,  since  effective  problem  representation  is  a  key  metacognitive  skill,  we  could 
use  the  KAT  called  Cognitive  Structure  Analysis  (Leddo  &  Cohen,  1988).  (This 
technique  grows  from  a  conceptual  model  called  Integrated  Knowledge  Structures  — 
INKS  —  described  by  Laskey  et  al.,  1990.)  Cognitive  Structure  Analysis  purports  to 
identify  the  knowledge  representations  the  expert  uses:  production  rules,  scripts, 
frames,  semantic  networks,  or  mental  models.  Not  only  would  this  KAT  be  useful  in 
identifying  the  expert’s  representation  of  the  problem,  it  might  also  describe  the 
expert’s  overarching  representation  of  the  problem-solving  process.  I  suspect  that 
most  scientists  have  a  mental  model  of  the  scientific  method  that  they  use  for  the 
everyday  conduct  of  scientific  activity. 

Several  metacognitive  skills  are  the  subject  of  psychological  research,  and  the 
criterion  variables  used  to  represent  these  skills  may  provide  a  means  of  assessment. 
Aircrew  coordination  and  communication,  for  example,  has  been  operationalized  as  - 
ratings  based  on  specific  behaviors,  e.g.,  the  discussion  of  potential  coordination 
problems  during  preflight  briefings  (Franz  et  al.,  1990).  Performance  in  games  or 
simulations  has  also  been  used  to  define  metacognitive  skills.  These  games  require 
coordination  between  two  or  more  team  members  for  superior  performance  (Bowers  et 
al.,  1992). 

Finally,  there  are  numerous  tests  available  for  the  assessment  of  reasoning  and 
problem  solving  ability.  Tests  of  diagnostic  ability  (e.g.,  troubleshooting)  are  also 
available. 

We  should  mention  a  special  assessment  technique  known  as  Career  Path 
Appreciation  (CPA;  Stamp,  1988).  CPA  was  developed  to  measure  the  level  of 
cognitive  complexity  that  a  member  of  an  organization  was  dealing  with  at  the  present 
time.  This  level  is  assumed  to  predict  later  career  development  according  to  Stratified 
Systems  Theory  (Jacobs  &  Jaques,  1987).  CPA  is  essentially  a  structured  interview 
that  focuses  on  a  respondent’s  general  approach  to  problems;  it  should  therefore  be 
well  adapted  for  the  assessment  of  metacognitive  skills.  However,  very  little 
information  is  available  on  the  technique  or  on  the  scoring  of  the  interview  protocols. 
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STICKY  WICKETS 


Although  this  is  a  rough  draft  of  the  conceptual  model,  we  have  some 
confidence  that  our  basic  approach  is  sound  and  promising.  I  would  like  to  mention 
two  issues  that  have  come  up,  not  because  they  present  problems  or  obstacles,  but 
because  they  represent  options  that  deserve  further  consideration  before  they  are 
discarded.  The  first  issue  has  to  do  with  levels  of  abstraction:  How  many  should  we 
consider?  We  have  described  the  conceptual  model  as  if  there  were  only  two  levels  of 
abstraction:  cognitive  skills  and  metacognitive  skills.  But  some  metacognitive  skills 
seem  more  abstract  than  others.  Sternberg,  for  example,  states,  “It  seems  likely  that 
the  solution-monitoring  metacomponent  controls  intercommunication  and 
interactivation  among  the  other  metacomponents,  and  there  is  a  certain  sense  In 
which  this  particular  metacomponent  might  be  viewed  as  a  ‘metametacomponent’” 
(Sternberg,  1984,  p.  171).  Flavell,  commenting  on  Sternberg’s  theory,  asks,  “...  are 
there  still-more-elementary  information  processes  that  several,  or  even  all,  of  his 
different  components  share  in  common?  ...  do  the  different  components  have  common 
subcomponents?”  (Flavell,  1984,  p.  206).  The  hinting  going  on  here  seems  directed 
toward  a  hierarchical  network  sort  of  arrangement,  with  highly  abstract  (generic) 
processes  (like  “solution-monitoring")  towards  the  top  and  relatively  concrete  (domain 
specific)  processes  (like  “turn  screws  clockwise  to  Install”)  towards  the  bottom. 

A  second  sticky  wicket  concerns  the  most  effective  way  to  represent  the 
influence  of  metacognitive  skills  on  human  performance.  We  have  used  a  flowchart 
approach,  which  has  the  advantage  of  showing  precisely  where  in  a  temporal  process 
a  given  metacognitive  skill  exerts  its  influence.  An  alternative,  which  has  been  used 
effectively  in  a  variety  of  applications,  is  a  layered  model.  Stratified  Systems  Theory 
(Jacobs  &  Jaques,  1987),  associated  with  the  Career  Path  Appreciation  assessment 
technique  discussed  above,  posits  seven  layers  (levels  of  work)  to  which  an  executive 
can  aspire.  Each  layer  has  different  tasks;  the  higher  layers  present  more  complex  - 
problems,  and  the  person  attempting  to  solve  the  problems  will  not  be  able  to  evaluate 
the  solution  for  ever  increasing  time  spans.  Thus,  at  the  highest  level,  the  chief 
executive  officer  of  a  major  corporation  solves  exceptionally  complex  problems  that  set 
the  company  on  a  course  of  action,  the  success  or  failure  of  which  will  not  become 
apparent  for  many  years.  People  at  each  level  depend  upon  the  skills  and  knowledge 
of  those  In  lower  layers  to  implement  their  plans  and  depend  upon  the  metacognitive 
activities  of  those  in  higher  layers  to  set  the  goals  and  objectives  for  their  relatively 
concrete  implementations.  The  seven  levels  are  further  divided  into  three  sets, 
defining  three  macroievels:  direct,  organizational,  and  executive.  We  are  obviously 
interested  in  the  highest  of  the  macroievels,  but  SST  provides  a  means  for  describing 
the  interrelationships  between  the  executive  level  and  the  organizational  (managers) 
and  the  direct  (workers  and  supervisors). 

A  similar  approach,  albeit  for  an  entirely  different  purpose,  is  the  Command  and 
Control  Reference  Model  (C2RM:  Mayk  &  Geiwitz,  1992).  C^RM  is  a  framework  for  the 
conceptualization,  design,  development,  and  operation  of  intelligent  C2  systems,  a 
common  language  for  interactions  that  promotes  open  design  and  interoperability. 
Patterned  after  the  Open  System  Interconnection  Reference  Model  (OSI  RM)  used  in 
the  telecommunications  industry,  it  describes  interactions  between  one  C^  system  and 
another  in  terms  of  seven  layers  of  functions  and  services  in  each  of  the  two  interacting 
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systems.  The  seven  layers  are  called  Application,  Presentation,  Session,  Transport, 
Networ^,  Link,  and  Physical.  The  Application  Layer  is  further  sublayered,  and  the 
C2RM  is  primarily  a  definition  and  exposition  of  these  seven  Application  sublayers. 
“The  highest  C2  Application  Layer  is  the  Conflict  Layer,  which  describes  the  Policies 
and  methods  used  by  Commanders  to  provide  Application  Services  in  the  form  of 
Mission  statements.  The  second  Application  Layer  is  the  Presentation  Layer,  which 
describes  the  Strategies  and  methods  used  by  Planners  to  provide  Application 
Seivices  in  the  form  of  Plans.  The  third  Application  Layer  is  the  Operation  Layer, 
which  describes  the  Tactics  and  methods  used  by  Controllers  to  provide  Application 
Services  in  the  form  of  Tasks.  The  fourth  Application  Layer  is  the  Procedure  Layer, 
which  describes  the  Schemas  and  methods  used  by  Agents  to  provide  Application 
Services  in  the  form  of  Jobs.  The  fifth  Application  Layer  is  the  Network  Layer,  which 
describes  the  Disciplines  and  methods  used  by  Administrators  to  provide  Application 
Services  in  the  form  of  Assignments.  The  sixth  Application  Layer  is  the  Link  Layer, 
which  describes  the  Techniques  and  methods  used  by  Coordinators  to  provide 
Application  Services  in  the  form  of  Transactions.  The  seventh  Application  Layer  is  the 
Asset  Layer,  which  describes  the  Instructions  and  methods  used  by  Operators  to 
provide  Application  Services  in  the  form  of  Packages"  (Mayk  &  Geiwitz,  1992,  p.  vlii). 

Although  the  C^RM  is  designed  to  describe  the  interactions  between  two  C^ 
systems  —  a  friendly  battalion  and  an  enemy  regiment,  or  a  computerized  mission 
planning  aid  such  as  the  Army’s  Mission  Control  System  and  the  enemy  regiment  —  it 
should  be  effective  for  the  description  of  the  C^  activities  of  a  single  individual  —  a 
Corps  commander,  for  example.  A  human  commander  is,  after  all,  a  C^  system  and,  in 
fact,  is  the  precise  C^  system  that  many  computerized  decision  aids  are  designed  to 
mimic  and  support.  The  advantages  of  modeling  executive  performance  with  the 
C2RM  remain  to  be  seen,  but  the  services  and  functions  of  each  Application  sublayer 
are  clearly  specified  and  their  interrelationships  worked  out.  The  highest  sublayer,  the 
Conflict  Layer,  seems  to  have  functions  that  are  clearly  metacognitive,  in  terms  of  our 
conceptual  model,  providing  the  overall  goals  and  mission  statements.  The  second 
sublayer,  the  Presentation  Layer,  uses  strategies  to  construct  plans  for  the  attainment 
of  mission  objectives.  Lower  layers  provide  supporting  actions.  The  C^RM  clearly 
could  be  used  to  describe  the  influence  of  metacognitive  skills  on  problem-solving 
performance,  a  possibility  we  will  explore  in  parallel  with  the  more  conventional  model 
building  described  in  the  body  of  this  paper. 
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